CPU-GPU Heterogeneous Systems Research Articles

Reuse distance-based shared LLC management mechanism for heterogeneous CPU-GPU systems

Reuse distance-based shared LLC management mechanism for heterogeneous CPU-GPU systems

Design Space Exploration for Partitioning Dataflow Program on CPU-GPU Heterogeneous System

Dataflow programming is a methodology that enables the development of high-level, parametric programs that are independent of the underlying platform. This approach is particularly useful for heterogeneous platforms, as it eliminates the need to rewrite application software for each configuration. Instead, it only requires new low-level implementation code, which is typically automatically generated through code generation tools. The performance of programs running on heterogeneous parallel platforms is highly dependent on the partitioning and mapping of computation to different processing units. This is determined by parameters that govern the partitioning, mapping, scheduling, and allocation of data exchanges among the processing elements of the platform. Determining the appropriate parameters for a specific application and set of architectures is a complex task and is an active area of research. This paper presents a novel methodology for partitioning and mapping dataflow programs onto heterogeneous systems composed of both CPUs and GPUs. The objective is to identify the program configuration that provides the most efficient way to process a typical dataflow program by exploring its design space. This is an NP-complete problem that we have addressed by utilizing a design space exploration approach that leverages a Tabu search meta-heuristic optimization algorithm driven by analysis of the execution trace graph of the program. The heuristic algorithm effectively identifies a solution that maps actors to processing units while improving overall performance. The parameters of the heuristic algorithm, such as the time limit and the proportion of neighboring solutions explored during each iteration, can be fine-tuned for optimal results. Additionally, the proposed approach allows for the exploration of solutions that do not utilize all hardware resources if it results in better performance. The effectiveness of the proposed approach is demonstrated through experimental results on dataflow programs.

A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous system

A perceptual and predictive batch-processing memory scheduling strategy for a CPU-GPU heterogeneous system

A Heterogeneous Parallel Computing Approach Optimizing SpTTM on CPU-GPU via GCN

Sparse Tensor-Times-Matrix (SpTTM) is the core calculation in tensor analysis. The sparse distributions of different tensors vary greatly, which poses a big challenge to designing efficient and general SpTTM. In this paper, we describe SpTTM on CPU-GPU heterogeneous hybrid systems and give a parallel execution strategy for SpTTM in different sparse formats. We analyze the theoretical computer powers and estimate the number of tasks to achieve the load balancing between the CPU and the GPU of the heterogeneous systems. We discuss a method to describe tensor sparse structure by graph structure and design a new graph neural network SPT-GCN to select a suitable tensor sparse format. Furthermore, we perform extensive experiments using real datasets to demonstrate the advantages and efficiency of our proposed input-aware slice-wise SpTTM. The experimental results show that our input-aware slice-wise SpTTM can achieve an average speedup of 1.310 × compared to ParTI! library on a CPU-GPU heterogeneous system.

Assessing Application Efficiency and Performance Portability in Single-Source Programming for Heterogeneous Parallel Systems

We analyze the performance portability of the skeleton-based, single-source multi-backend high-level programming framework SkePU across multiple different CPU–GPU heterogeneous systems. Thereby, we provide a systematic application efficiency characterization of SkePU-generated code in comparison to equivalent hand-written code in more low-level parallel programming models such as OpenMP and CUDA. For this purpose, we contribute ports of the STREAM benchmark suite and of a part of the NAS Parallel Benchmark suite to SkePU. We show that for STREAM and the EP benchmark, SkePU regularly scores efficiency values above 80% and in particular for CPU systems, SkePU can outperform hand-written code.

Portrait: A holistic computation and bandwidth balanced performance evaluation model for heterogeneous systems

Accelerators are widely used in specific domains ranging across deep learning, streaming computation and database query. To enable an accelerator, it has to be attached to a primary controller, typically, a CPU. This kind of CPU-accelerator heterogeneous systems is the mainstream of current computer systems. In CPU-accelerator heterogeneous systems, customized optimization for accelerators boosts performance and energy efficiency. However the bandwidth contention of CPU-accelerator interconnection and hardware hazard between multiple tasks on accelerators significantly bottleneck the designed performance. On one hand, limited interconnected bandwidth resource causes bandwidth contention in task offloading from CPU to accelerators. On the other hand, limited hardware resources on accelerators cause hardware hazard during task execution. To take fully advantage of designed computing power in CPU-accelerator heterogeneous system, it is necessary to mitigate these kinds of contention. However, it is hard for programmers and users to solve the contention because of the complexity of both computing tasks and system behaviors. In state-of-the-art, CPU–GPU heterogeneous system have been sufficiently studied. But CPU–FPGA heterogeneous systems are seldom comprehensive analyzed. To help to address this problem, we propose a holistic profiling system, Portrait, to help to model both computation and bandwidth resource in CPU-accelerator heterogeneous system and quantify bandwidth requirement and execution time of given tasks. The experiment shows that Portrait increases the accuracy of the bandwidth requirement up to 97.71% on average, which is 1.95x compared with the state-of-the-art. It also provides a more accurate of computation latency compared to the state-of-the-art that failed to evaluate accelerator behaviors. And it increases accuracy of task execution latency over 97.47% on average. Additionally, based on precise profiling of CPU-accelerator heterogeneous system, Portrait could help task scheduling to mitigate bandwidth contention and hardware hazard more effectively to improve system throughput.

An asynchronous and parallel row-wise compressed SpMV kernel on heterogeneous CPU-GPU architectures

Sparse matrix vector multiplication (SpMV) is a fundamental and important algebra used extensively in various fields, such as machine learning, data mining and numerical simulation. Accelerating SpMV benefits the corresponding real-world applications. Meanwhile, heterogeneous CPU-GPU architectures are inevitable to realise high-performance computing. Therefore, this paper designs an asynchronous parallel row-wise compressed SpMV kernel by utilising the heterogeneous CPU-GPU architectures to accelerate applications that use SpMV. First, according to the memory access and control flow of SpMV and the architectural features of the heterogeneous CPU-GPU systems, the heterogeneous parallelisation of the row-wise compressed SpMV based on CSR format is designed. Next, an asynchronous method is designed for the parallel compressed SpMV kernel to create an additional level of parallelism in SpMV execution. The proposed SpMV kernel can obtain up to 8.97% improvement from the asynchronous method on the heterogeneous CPU-GPU architecture and perform the nearly linear speedup across GPU threads.

An optimized workflow scheduling algorithm on CPU-GPU heterogeneous systems

Aiming at the workflow scheduling problem on CPU-GPU heterogeneous systems, this article proposes a workflow scheduling algorithm that optimizes task priority and processor selection phase. This article focuses on the relationship between processor processing time and task acceleration ratio, and uses optimistic finish time table. This article estimates each task’s communication cost and computation cost on the basis of each task’s acceleration ratio ri. Then the algorithm calculates the task’s priority ranku based on communication cost and computation cost. In processor selection phase, this article takes the Earliest Finish Time (EFT) difference between the processor with the fastest task and the processor selected based on Heterogeneous Earliest Finish Time (HEFT) algorithm as the judgment condition k. If the difference between the earliest completion time of all child tasks is greater than k, the algorithm selects the processor selected by the HEFT algorithm; otherwise, the algorithm selects another one. This article uses the computational shared facility’s kernel timer for simulation experiment, which bases on the University of Manchester’s high performance computing cluster. Based on the simulation results, the proposed algorithm can reduce the maximum completion time and energy consumption, and improve the acceleration ratio. Compared with the HEFT algorithm, the task maximum completion time and energy consumption are reduced by 10% and 5% respectively, and the acceleration ratio is improved by 3%.

СПОСІБ РОЗРОБКИ ІЗОЕФЕКТИВНОЇ ГЕТЕРОГЕННОЇ СИСТЕМИ НА ОСНОВІ МАШИННОГО НАВЧАННЯ ДЛЯ ЗАДАЧІ ДИСКРЕТНОГО ПЕРЕТВОРЕННЯ ФУР’Є

In this paper, the isoefficiency of MPP systems and heterogeneous CPU-GPU systems on the problem of discrete Fourier transform is considered. The development of parallel applications as its goal can not only reduce execution time, but also provide opportunities to solve problems of a larger dimension. The peculiarity of algorithm parallelization includes the efficient use of hardware while increasing the dimension of the problem is an important characteristic of parallel computing. However, currently heterogeneous systems have not been researched extensively to determine isoefficiency characteristics and build application-specific systems around said method, although there are articles that show potential using isoefficiency to design the system and using heterogeneous approach to accelerate performance of different tasks. Discrete Fourier Transform algorithm lets build systems that discretize analogue and digital signals and it can serve as a benchmark to test different systems. Algorithms suited for MPP systems can use analytical approach to find out issoefficiency function and to determine how scaling the system or changing the size of the task will change its performance metrics. One of the most popular approaches to linking up processing units in MPP systems is using hypercube topology. MPP system that is connected using this topology will be analyzed. CPU-GPU heterogeneous system will be analyzed using an approach based on polynomial regression. Due to the nature of heterogeneous systems, analytic approach used in MPP system is impossible. Predictive model based on polynomial regression will use modelling results from using CPU and GPU separately to estimate how much time it will take for heterogeneous system to finish the task. To ensure accuracy of the experiment, several systems will be used to model the task. Using this approach, resulting issoefficient heterogeneous system will be analyzed using performance metrics s

Optimizing the LINPACK Algorithm for Large-Scale PCIe-Based CPU-GPU Heterogeneous Systems

There is a widening gap between GPU and other components (CPU, PCIe bus and communication network) in heterogeneous parallel system. The gap forces us to orchestrate cooperative execution among these components much more carefully than ever before. By taking the LINPACK benchmark as a case study, this article proposes a fine-grained pipelining algorithm on large-scale CPU-GPU heterogeneous cluster systems. First, we build an algorithmic model that reveals a new approach to GPU-centric and fine-grained pipelining algorithm design. Then, we present four model-driven pipelining algorithms that incrementally squeeze bubbles in the pipeline so that it is occupied by more useful floating-point calculations. The algorithms are implemented on both the AMD and NVIDIA GPU platforms. The finally optimized LINPACK program achieves 107 PFlops on 25, 600 GPUs (70 percent floating-point efficiency). Several insights have been drawn to suggest tradeoff of algorithm design, programming support, and architecture design.

Parallel implementation of the four-dimensional lattice spring model on heterogeneous CPU-GPU systems

As a newly developed computational method, the four-dimensional lattice spring model (4D-LSM) is computationally intensive due to the introduction of extra-dimensional interactions. In this work, the 4D-LSM is parallelized to fully utilize the available computational resources of modern computers, namely, the multi-core CPU and the GPU. To utilize computing power of the multi-core CPU, OpenMP with a fork-join scheme is used to assign computational tasks to different CPU threads, whereas CUDA, with a granular computing scheme, is adopted to assign computations to thousands of GPU threads. A domain decomposition with a data communication scheme is proposed to utilize both the multi-core CPU and the GPU. The influence of digital precision and hardware on the parallel computing performance of the 4D-LSM are investigated through a number of numerical examples including elastic deformation, elastic bulking and dynamic fracturing. Finally, the multi-core CPU 4D-LSM is used to solve a crack propagation problem and is compared with existing experimental and numerical results.

Multi-GPU immersed boundary method hemodynamics simulations

Large-scale simulations of blood flow that resolve the 3D deformation of each comprising cell are increasingly popular owing to algorithmic developments in conjunction with advances in compute capability. Among different approaches for modeling cell-resolved hemodynamics, fluid structure interaction (FSI) algorithms based on the immersed boundary method are frequently employed for coupling separate solvers for the background fluid and the cells within one framework. GPUs can accelerate these simulations; however, both current pre-exascale and future exascale CPU-GPU heterogeneous systems face communication challenges critical to performance and scalability. We describe, to our knowledge, the largest distributed GPU-accelerated FSI simulations of high hematocrit cell-resolved flows with over 17 million red blood cells. We compare scaling on a fat node system with six GPUs per node and on a system with a single GPU per node. Through comparison between the CPU- and GPU-based implementations, we identify the costs of data movement in multiscale multi-grid FSI simulations on heterogeneous systems and show it to be the greatest performance bottleneck on the GPU.

Asynchronous Processing for Latent Fingerprint Identification on Heterogeneous CPU-GPU Systems

Latent fingerprint identification is one of the most essential identification procedures in criminal investigations. Addressing this task is challenging as (i) it requires analyzing massive databases in reasonable periods and (ii) it is commonly solved by combining different methods with very complex data-dependencies, which make fully exploiting heterogeneous CPU-GPU systems very complex. Most efforts in this context focus on improving the accuracy of the approaches and neglect reducing the processing time. Indeed, the most accurate approach was designed for one single thread. This work introduces the fastest methodology for latent fingerprint identification maintaining high accuracy called Asynchronous processing for Latent Fingerprint Identification (ALFI). ALFI fully exploits all the resources of CPU-GPU systems using asynchronous processing and fine-coarse parallelism for analyzing massive databases. Our approach reduces idle times in processing and exploits the inherent parallelism of comparing latent fingerprints to fingerprint impressions. We analyzed the performance of ALFI on Linux and Windows operating systems using the well-known NIST/FVC databases. Experimental results reveal that ALFI is in average 22x faster than the state-of-the-art algorithm, reaching a value of 44.7x for the best-studied case.

Leveraging access mode declarations in a model for memory consistency in heterogeneous systems

On a system that exposes disjoint memory spaces to the software, a program has to address memory consistency issues and perform data transfers so that it always accesses valid data. Several approaches exist to ensure the consistency of the memory accessed. Here we are interested in the verification of a declarative approach where each component of a computation is annotated with an access mode declaring which part of the memory is read or written by the component. The programming framework uses the component annotations to guarantee the validity of the memory accesses. This is the mechanism used in VectorPU, a C++ library for programming CPU-GPU heterogeneous systems. This article proves the correctness of the software cache-coherence mechanism used in VectorPU. Beyond the scope of VectorPU, this article provides a simple and effective formalization of memory consistency mechanisms based on the explicit declaration of the effect of each component on each memory space. The formalism we propose also takes into account arrays for which a single validity status is stored for the whole array; additional mechanisms for dealing with overlapping arrays are also studied.

HyPar: A divide-and-conquer model for hybrid CPU–GPU graph processing

Efficient processing of graph applications on heterogeneous CPU–GPU systems require effectively harnessing the combined power of both the CPU and GPU devices. This paper presents HyPar, a divide-and-conquer model for processing graph applications on hybrid CPU–GPU systems. Our strategy partitions the given graph across the devices and performs simultaneous independent computations on both the devices. The model provides a simple and generic API, supported with efficient runtime strategies for hybrid executions. The divide-and-conquer model is demonstrated with five graph applications and using experiments with these applications on a heterogeneous system it is shown that our HyPar strategy provides equivalent performance to the state-of-art, optimized CPU-only and GPU-only implementations of the corresponding applications. When compared to the prevalent BSP approach for multi-device executions of graphs, our HyPar method yields 74%–92% average performance improvements.

A survey on techniques for cooperative CPU-GPU computing

Graphical Processing Unit provides massive parallelism due to the presence of hundreds of cores. Usage of GPUs for general purpose computation (GPGPU) has resulted in execution speedup and energy efficiency. In addition, the modern CPUs also possess multiple cores, with enormous computational power. In general, the current programming frameworks for CPU-GPU heterogeneous computing system do not facilitate the efficient utilization of computational resources so as to further improve the performance and reduce the energy consumption. Several researches have been carried out to improve the resource utilization and reduce the energy consumption of heterogeneous systems by cooperatively performing the computation on both multicore CPUs and GPUs. In this survey paper we review the various techniques available for CPU-GPU cooperative computing.

An orchestrated NoC prioritization mechanism for heterogeneous CPU-GPU systems

In a heterogeneous CPU-GPU multicore system that contains various types of computation units as well as on-chip storage units, the on-chip interconnection network is a critical shared resource responsible for sending coherence and memory traffic. On-chip traffic originated from or designated to different components has different performance and throughput requirements. A naive or un-optimized traffic prioritization mechanism usually results in suboptimal system performance. In this work, we quantify the performance/throughput requirements for both CPU and GPU applications, abstract critical information, and propose a network prioritization mechanism which effectively coordinates the on-chip traffic to improve overall system performance.

A Hybrid Parallel Spatial Interpolation Algorithm for Massive LiDAR Point Clouds on Heterogeneous CPU-GPU Systems

Nowadays, heterogeneous CPU-GPU systems have become ubiquitous, but current parallel spatial interpolation (SI) algorithms exploit only one type of processing unit, and thus result in a waste of parallel resources. To address this problem, a hybrid parallel SI algorithm based on a thin plate spline is proposed to integrate both the CPU and GPU to further accelerate the processing of massive LiDAR point clouds. A simple yet powerful parallel framework is designed to enable simultaneous CPU-GPU interpolation, and a fast online training method is then presented to estimate the optimal decomposition granularity so that both types of processing units can run at maximum speed. Based on the optimal granularity, massive point clouds are continuously partitioned into a collection of discrete blocks in a data processing flow. A heterogeneous dynamic scheduler based on the greedy policy is also proposed to achieve better workload balancing. Experimental results demonstrate that the computing power of the CPU and GPU is fully utilized under conditions of optimal granularity, and the hybrid parallel SI algorithm achieves a significant performance boost when compared with the CPU-only and GPU-only algorithms. For example, the hybrid algorithm achieved a speedup of 20.2 on one of the experimental point clouds, while the corresponding speedups of using a CPU or a GPU alone were 8.7 and 12.6, respectively. The interpolation time was reduced by about 12% when using the proposed scheduler, in comparison with other common scheduling strategies.

Dual buffer rotation four-stage pipeline for CPU–GPU cooperative computing

Accelerators such as GPUs have become popular general-purpose computing device in the field of high-performance computing. With the boosting ability of storage and computation, it is very important to solve the complex scientific and engineering problems on CPU–GPU heterogeneous system in the big data era. Now the compute-intensive problems have been successfully solved using CPU–GPU cooperative computing. However, it is difficult to handle large-scale data-intensive problems, especially for those limited by GPU device memory. In this paper, the dual buffer rotation four-stage pipeline (DBFP) mechanism is proposed for CPU–GPU cooperative computation to efficiently handle data-intensive problems, which need larger memory than that of a single GPU. The data block partition-based pipeline computing strategy is designed on top of the DBFP mechanism. On the one hand, it breaks out the bottleneck of limited GPU device memory. On the other hand, it explores high-performance computing of CPU and GPU with data transfer and computation overlap. Furthermore, it is easy to extend the DBFP mechanism on the heterogeneous system equipped with multiple GPUs and achieve high resource utilization. The results show that it can achieve 99 and 90% of theoretical performance for dense general matrix multiplication on one GPU and two GPUs respectively with Nvidia GTX480 or K40 GPUs. It also enables K-means and T-nearest-neighbor algorithms to process larger datasets, which used to be limited by the GPU device memory. We achieve nearly 1.9-fold performance gains by dynamic task scheduling on two GPUs when the performance bottleneck is GPU computing or data transmission.

PARALLEL ARTIFICIAL NEURAL NETWORK APPROACHES FOR DETECTING THE BEHAVIOUR OF EYE MOVEMENT USING CUDA SOFTWARE ON HETEROGENEOUS CPU-GPU SYSTEMS

Eye movement behaviour is related to human brain activation either during asleep or awake. The aim of this paper is to measure the three types of eye movement by using the data classification of electroencephalogram (EEG) signals. It will be illustrated and train using the artificial neural network (ANN) method, in which the measurement of eye movement is based on eye blinks close and open, moves to the left and right as well as eye movement upwards and downwards. The integrated of ANN with EEG digital data signals is to train the large-scale digital data and thus predict the eye movement behaviour with stress activity. Since this study is using large-scale digital data, the parallelization of integrated ANN with EEG signals has been implemented on Compute Unified Device Architecture (CUDA) supported by heterogeneous CPU-GPU systems. The real data set from eye therapy industry, IC Herbz Sdn Bhd was carried out in order to validate and simulate the eye movement behaviour. Parallel performance analyses can be captured based on execution time, speedup, efficiency, and computational complexity.